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OVERVIEW 


Successful  commercialization  of  new  technologies  requires  good 
communication  and  information  flow  among  different  corporate  functions. 
Good  communication  means  bringing  the  multiple,  and  sometimes 
conflicting,  perspectives  from  R&D,  engineering,  operations,  and 
management  to  bear  on  the  project.  These  groups  can  share  information 
and  experiences.  At  least  as  important,  information  obtained  from 
previous  efforts  should  be  shared.  ’'Lessons  learned”  from  the  operation 
of  pilot  plants  to  the  startup  of  pioneer  commercial  plants  need  to  be 
remembered. 

Our  research  suggests  that  many  firms  fail  to  learn  effectively 
from  past  experience  or  to  communicate  adequately  across  these  corporate 
functions,  or  divisions.  Research  and  development,  along  with  operating 
divisions,  are  often  isolated  from  project  design  and  execution.  R&D 
often  won't  know  how  well  the  process  they  developed  actually  worked; 
operators  will  have  little  voice  in  the  project  until  they  have  to  take 
it  over  and  make  it  work.  Furthermore,  useful  experiences  are 
frequently  lost  and  forgotten  as  a  direct  result  of  the  way  companies 
keep  and  use  information.  This  reflects  corporate  information  policies 
and  practices.  Communication  often  fails  because  the  substance  has  been 
lost.  In  other  words,  not  remembering  what  happened  makes  it  hard  to 
communicate . 

GOOD  COMMUNICATION  AND  INFORMATION  FLOW  ESSENTIAL 

For  a  commercialization  effort  to  be  successful,  R&D,  engineering, 
operations,  and  management  must  all  communicate  with  each  other.  There 
should  be  feedback  among  all  functions  on  an  ongoing  basis,  illustrated 
in  Chart  1,  so  that  each  benefits  from  the  experiences  of  the  others  in 
ways  that  are  useful  to  each  division  improving  the  way  it  carries  out 
its  responsibilities.  We  will  return  to  this  seemingly  obvious--but 
often  ignored--point  later. 


Chart  1  --  Good  Communication  and  Information  Flow  Essential 

PIONEER  PLANT  CRITICAL  STEP  IN  COMMERCIALIZATION 

The  process  of  commercializing  a  new  technology  involves  multiple 
stages  before  the  pioneer  plant  is  actually  built,  if  it  ever  is.  These 
are  illustrated  in  Chart  2.  Each  step  generates  information  that 
improves  our  understanding  of  the  technology.  It  may  also  raise  new 
questions.  Beginning  with  laboratory  and  bench  scale  testing  and 
evaluation,  we  must  decide  whether  or  not  to  proceed  with  development. 
Each  step  also  becomes  more  costly,  so  the  decision  must  be  made  more 
carefully.  The  next  step  might  be  to  build  a  process  development  unit, 
followed  by  an  integrated  pilot  plant.  Development  may  be  stopped  after 
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Chart  2  --  Pioneer  Plant  Critical  Step  in  Commercialization 

In  some  cases,  a  firm  will  decide  to  proceed  with  a  commercial- 
scale  facility.  Some  technical  uncertainties  still  exist  that  only  a 
full-sized  operating  unit  can  answer.  Building  a  pioneer  plant  involves 
a  great  deal  more  money,  of  course,  compared  with  earlier  expenditures 
during  process  development.  Our  discussion  will  focus  on  how  company 
information  practices  and  project  communication  affect  the  project's 
outcomes  and  whether  or  not  vital  lessons  are  remembered. 


Types  of  issues  addressed  by  pioneer  unit 

As  the  first  commercial  demonstration  of  a  new  technology,  the 
pioneer  plant  can  reduce  many  remaining  uncertainties  significantly. 
These  issues  concern  capital  as  well  as  O&M  costs,  schedule  problems, 
technical  performance,  startup  difficulties,  design  failures,  product 
quality,  environmental  impacts,  etc.  The  f irst-of-a-kind  unit  can 
suggest  possible  process  and  design  improvements --if  the  problems  and 
their  solutions  are  recallable  when  needed  later.  The  pioneer  plant  is 
the  potential  source  of  invaluable  learning. 

Who  needs  to  know? 

The  lessons  learned  during  design,  construction,  startup,  and  early 
operation  of  a  pioneer  plant  are  useful  to  each  functional  unit  for 
somewhat  different  purposes.  R&D  needs  to  know  what  happened  during  the 
pioneer  project  to  improve  its  basic  process  understanding  and  to  help 
guide  future  research  efforts.  Engineering  needs  to  understand  how  well 
their  design  solutions  performed,  and  how  future  designs  might  be 
adapted  to  avoid  problems  that  arose  before,  during,  and  after  startup. 
Operations  should  understand  the  concerns  of  R&D  and  engineering  in 
developing,  designing  and  building  the  plant,  and  feed  back  to  them 
information  about  difficulties  encountered  during  and  after  startup. 
Corporate  management  needs  this  type  of  information  to  guide  future 
research  and  development  allocations,  and  to  evaluate  the  bottom  line: 
profitability.  This  information  is  also  necessary  in  deciding  whether 
to  build  additional  plants  of  the  same  type. 

How  do  they  know? 

These  groups  can  only  learn  from  the  pioneer  project  experiences  if 
they  are  captured  and  preserved  in  such  a  way  that  the  information  is 
later  available  for  reference.  They  can  also  learn  from  each  other 
during  the  project  (and  subsequent  ones)  through  close  interaction  with 
each  other.  Such  information  can  flow  more  easily  when  teams  composed 
of  representatives  from  R&D,  engineering,  and  operations  are  made 
jointly  responsible  for  project  execution.  This  practice  places  people 
with  diverse  perspectives  at  the  project  management  level,  and  reduces 
the  likelihood  of  unexpected  problems  arising  during  execution. 
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Of  course,  information  is  only  valuable  if  it  is  used.  The  more 
such  information  is  used,  the  more  likely  that  it  will  be  captured  and 
preserved  in  the  first  place,  and  thus  the  greater  stake  other  users 
will  have  in  seeing  that  the  information  is  maintained  and  used  over 
time . 

In  sum,  for  pioneer  projects  to  be  useful,  firms  (not  just 
individuals)  must  know  what  happened,  remember  it,  share  it  across 
divisions  or  functions,  and  use  it. 

Draw  on  results  of  recent  Rand  research 

This  discussion  draws  together  the  results  of  three  Rand 
Corporation  studies.  The  Pioneer  Plants  Study  (PPS)  examined  cost, 
schedule,  and  performance  problems  commonly  encountered  by  first-of- 
a-kind  chemicd1  process  plant  projects.1  The  analysis  was  based  on 
proprietary  data  on  over  60  projects  provided  by  more  than  40  firms  in 
the  oil,  chemical,  and  minerals  processing  industry.  Another  study 
assessed  the  information  policies  and  practices  of  19  medium  to  large 
firms  in  the  process  industries.2  The  research  drew  on  our  experiences 
in  collecting  project  information,  and  on  the  results  of  40  interviews 
with  corporate  personnel.  A  third  study  analyzed  the  effects  of  certain 
management  practices  on  project  outcomes--cost ,  schedule,  and 
performance,  especially  for  pioneer  plant  projects.3  The  effect  of 
using,  for  example,  project  management  teams  that  included  R&D  and 
operations  personnel,  was  quantitatively  evaluated. 


‘Merrow,  E.  W.,  K.  E.  Phillips,  and  C.  W.  Myers,  Understanding  Cost 
Growth  and  Performance  Shortfalls  in  Pioneer  Process  Plants,  The  Rand 
Corporation,  R- 2569 -DOE,  September  1981.  Myers,  C.  W. ,  M.  R.  Devey,  and 
R.  F.  Shangraw,  Under  stand ing  Process  Plant  Schedule  Slippage  and 
Startup  Costs,  The  Rand  Corporation,  R-3215-PSSP,  forthcoming. 

2Myers,  C.  W. ,  and  R.  Y.  Arguden,  Capturing  Pioneer  Plant 
Exper ience :  Impl icat ions  for  Synfuels  Projects ,  The  Rand  Corporation, 
N-2063-SFC,  January  1984. 

3Myers,  C.  W.,  and  M.  R.  Devey,  How  Management  Practices  Can  Affect 
Project  Outcomes :  An  Explorat ion  of  the  PPS  Database ,  The  Rand 
Corporation,  N-2196-SFC,  August  1984. 
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WHY  IS  PIONEER  PROJECT  INFORMATION  IMPORTANT? 

Pioneer  project  information  proved  to  be  very  important  to  firms 
contemplating  designing,  building,  and  operating  a  follow-on  (i.e., 
second-of -a-kind)  plant.  In  summary  form,  Chart  3  shows  the  kinds  of 
questions  that  companies  ask  about  the  pioneer  project--even  if  they 
didn't  build  it  themselves.  They  particularly  want  to  know  what  it 
cost,  how  long  it  took,  how  well  it  worked,  and  what  problems  occurred. 
Companies  then  proceed  to  ask  themselves:  Can  it  be  done  better?  How 
can  the  design  of  the  next  plant  be  improved--to  make  startup,  for 
instance,  much  smoother?  What  uncertainties  remain?  Firms  turn  to  the 
experience  of  the  pioneer  plants  to  help  answer  these  questions. 


UNDERSTAND 
“WHAT  HAPPENED" 


•  What  did  it  cost? 

•  How  long  did  it  take? 

•  How  well  did  it  work? 

•  What  problems 


occurred? 


UNDERSTAND 
HOW  TO  DO  IT  AGAIN 

Can  it  be  done  better? 

What  uncertainties  remain? 


FOLLOW-ON  PROJECT 
INVESTMENT  AND 
DESIGN  DECISIONS 

•  What  will  it  cost? 

•  How  should  it  be 

designed? 

•  Can  we  do  it? 

•  How  good  is  our 


information? 


Chart  3  --  Why  is  Pioneer  Project  Information  Important? 
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Goals  of  retrospective  evaluation 

The  types  of  data  needed  for  cross-project  quantitative  analyses 
may  not  be  identical  to  those  any  one  firm  might  use  to  make  decisions 
about  building  later  plants.  In  general,  of  course,  past  project 
experience  is  helpful  in  making  these  decisions.  Such  information  can 
be  used  to  improve  the  firm's  estimating  capability,  to  compare  outcomes 
across  different  projects,  and  improve  design,  execution,  and  operation 
of  subsequent  plant  development  projects.  The  Pioneer  Plants  Study 
sought  to  understand  why  many  pioneer  plants  cost  more,  take  longer,  and 
perform  worse  than  expected.  In  doing  so,  the  PPS  developed  tools  for 
evaluating  past  as  well  as  proposed  projects.  In  this  sense,  the 
information  requirements  are  closely  parallel. 

Means  of  retrospective  evaluation 

The  Pioneer  Plants  Study  points  to  how  f irst-of -a-kind  projects  can 
be  compared.  Retrospective  evaluation  requires  two  steps: 

"normalizing"  data  across  projects,  and  then  evaluating  and  comparing 
specific  projects.  Data  are  normalized  by  comparing  different 
technologies  and  projects  on  as  common  a  basis  as  possible, 
differentiating  project-specific  versus  technology-specific  outcomes. 
That  is,  the  analyst  asks  whether  problems  in  the  project  were  due  to 
the  technology  choice  itself  or  to  other  conditions  relating  to  how  the 
project  was  managed  or  executed.  To  some  extent,  every  project  or 
technology  is  unique.  Differences  in  site  or  market  conditions,  owners, 
contractors,  etc.  may  appear  overwhelming  and  tend  to  inhibit  cross¬ 
technology  and  cross -pro j ect  comparisons. 

One  goal  is  to  be  able  to  abstract  from  the  specific  project 
details  in  order  to  draw  general  lessons  about  how  later  projects  should 
be  managed  and  executed.  A  second  goal  is  to  provide  information  that 
allows  decisionmakers  to  separate  promising  technologies  from 
unpromising  ones.  For  each  project,  the  nonrepeating  costs  and  problems 
must  be  identified  and  put  aside.  To  realistically  evaluate  the 
expectations  for  follow-on  projects,  it  is  especially  important  to 
understand  how  the  pioneer  project's  cost,  schedule,  and  performance 
varied  from  what  was  expected. 
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Information  needed  from  pioneer  projects 

Chart  4  summarizes  the  information  needed  from  the  pioneer  project. 
There  are  several  critical  cost  questions: 

•  What  was  the  total  capital  cost? 

•  Of  that  total,  what  was  the  particular  cost  of  the  first-of- 
a-kind  unit? 

•  How  much  did  it  cost  to  start  the  plant  up? 

•  How  much  does  it  cost  to  operate  and  maintain  the  plant? 


i 


License^ 


CRITICAL 

IMPORTANT 

Process  data 

Design  issues  and 

•  Yields/quality 

philosophy 

•  H&M  balances 

Environmental 

attainment 

Costs 

Equipments  specs/ 

•  Capital 

•  Pioneer  unit 

■vendors 

•  Startup 

'"secondary1 

•  O&M 

l  1 

'  Schedule  1 

Performance 

'  Estimates 

•  Actual  and  reliability 

•  Startup  problems/resolution 

•  Pioneer  unit 

Cost/schedule/performance 

problems 

i_  j 

Chart  4  --  Information  Needed  from  Pioneer  Projects 
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Information  on  the  plant's  performance  and  reliability  is  equally 
essent ial . 

•  What  were  the  startup  problems? 

•  How  were  they  resolved? 

•  How  well  did  the  new  technology  process  units  work? 

Another  item  on  the  critical  list  is  problems  in  cost,  schedule,  and 
performance.  This  information  is  important  to  interpret  what  happened 
with  the  pioneer  project:  if  it  differed  significantly  in  cost, 
schedule,  or  performance  from  what  they  expected  would  happen,  why  the 
problems  occurred,  how  they  dealt  with  those  problems,  and  what  they 
cost.  This  information  is  essential  to  interpret  what  happened  and  to 
understand  what  to  do  next  time. 

Cost  data  needed  after  mechanical  completion 

Some  data  on  project  cost  are  needed  after  mechanical  completion. 
These  are  illustrated  in  Chart  5.  Two  kinds  of  costs  are  incurred  at 
that  point:  capital  costs  of  startup,  and  then  the  expensed  costs  for 
starting  up  and  for  operating  and  maintaining  the  plant.  A  knowledge  of 
total  startup  costs  imparts  a  sense  of  how  problematical  the  project 
was,  and  whether  the  follow-on  project  can  reduce  those  costs.  For  many 
processes,  net  operating  and  maintenance  costs  are  as  important  as  the 
total  capital  cost.  Pro ject -spec i f ic  information  is  not  needed  from  the 
pioneer  plant,  however.  This  includes  the  costs  of  feedstock,  which  can 
overwhelm  the  OiM  costs;  royalties  for  the  technology;  and  production 
taxes.  These  are  all  site-,  project-,  or  market-specific  costs. 

Startup  data  needed 

Startup  and  early  operations  provide  extremely  important 
information  because  they  involve  both  cost  and  performance.  This 
information  is  useful  to  R&D  and  engineering  because  it  can  help  answer 
the  following  types  of  questions: 


Capital 


Operating 

and 

maintenance 


Total 

startup 

costs 


Net 

O&M  costs 


Feedstock 

costs 


Can  follow-on 
project  avoid 
or  reduce? 


Critical  to 
follow-on  project 
economics 


- 1 _ 

’N 

Project- 

- 1 - 

specific 

Production 

taxes  ' 


Chart  5  --  Cost  Data  Needed  After  Mechanical  Completion 

•  Are  problems  liked  to  the  technolcgy  or  poor  project  execution? 

•  Can  they  be  avoided  in  a  follow-on  project? 

•  How  should  the  follow-on  plant  design  be  modified? 

•  Is  further  R&D  warranted;  in  what  areas? 

In  fact,  startup,  more  than  anything  else,  was  reported  to  us  as 
providing  the  key  information  that  enables  the  analyst  to  distinguish 
technology  problems  from  problems  with  project  execution.  Understanding 
what  happened  during  plant  startup  is  therefore  critical  information  for 
R&D,  engineering,  operations,  and  management.  The  first  one  to  three 
years  of  initial  operations--the  numbers  commonly  reported  to  us--appear 
to  be  long  enough  to  provide  a  reasonable  handle  on  how  well  the 
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technology  performs,  how  reliable  it  is,  and  what  the  operating  and 
maintenance  costs  are. 

The  importance  of  specific  startup  information  is  often  overlooked. 
When  they  are  collected  and  preserved,  these  data  are  typically  kept  by 
the  operating  division  or  at  the  plant.  They  are  not  usually  shared 
with  the  process  developers,  designers,  or  others  in  the  same  firm. 

Many  problems  go  unrecognized,  and  their  solutions  unreported,  as  a 
result . 

PPS  DATA  COLLECTION  PROBLEMS 

Over  40  companies  have  participated  in  work  related  to  the  Pioneer 
Plants  Study  during  the  past  6  years  by  providing  the  types  of 
information  discussed  above.  Not  all  companies  contacted,  or  that 
expressed  interest  could  participate,  however.  The  reasons  are 
summarized  in  Chart  6.  About  20  firms  declined  to  participate  because 
of  lack  of  interest  or  concerns  over  data  sensitivity.  But  more  than 
two  dozen  firms  reported  either  that  they  did  not  keep  data  on  projects, 
or  that  the  data,  if  they  did  have  them,  were  too  hard  to  find  or 
transfer.  In  other  words,  well  over  half  the  companies  that  did  not 
participate  could  not  participate  because  they  did  not  have  the  relevant 
data--even  for  a  single  project. 

We  even  encountered  data  availability  problems  for  the  companies 
that  gave  us  reasonably  complete  information  on  at  least  one  of  their 
plants.  Chart  7  displays  some  of  the  key  information  categories  that  we 
collected,  and  the  proportion  of  plants  (or  estimates)  that  had 
complete,  analyzable  information.  Particularly  troubling,  given  their 
importance  to  retrospective  and  prospective  project  evaluations,  are  the 
information  gaps  for  process  development  units  and  startup.  Only  about 
a  third  of  the  firms  could  provide  process  development  information  on 
even  the  costs  and  dates  of  testing  facilites,  pilot  plants,  bench- 
scale  tests,  and  so  on,  that  were  relied  on  in  designing  the  commercial 
plant.  And  of  the  plants  that  had  problems  with  startup,  only  about  one 
third  could  describe  them  and  list  the  causes  and  corrective  measures 
taken.  This  implies  that  R&D  departments  are  not  keeping  good  records 
on  development  projects. 


Not 

interested 


No  data 
§j  kept  ii 


Chart  6  —  Why  Some  Firms  Did  Not 


Participate  in  PPS 


ACTUAL  COSTS 
Capital  costs 
Breakdown 
Startup  costs 
Manyears 

ESTIMATED  COSTS 
Plants  with  2  or  more 
Useable  qptimates 
Breakdown 
Startup  costs 
Dollar  basis 
Definition  basis 
PERFORMANCE 
First  1 8  months 
Design  issues 
Startup  problems/causes 
Process  development  information 
SCHEDULE 
Planned  time  by  phase 
Actual  time  by  phase 


Chart  7 


PPS  Data  Availability  Problems 
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In  sum,  the  information  that  firms  keep  from  pioneer  projects  is  by 
no  means  complete,  even  among  firms  that  wish  to  be  reasonably  thorough 
about  it. 

INFORMATION  KEPT  VARIES--EVEN  FOR  CRITICAL  DATA 

It  is  especially  troubling  to  find  such  information  gaps  concerning 
data  that  the  company  reported  to  us  as  being  critical  to  their 
evaluation  needs.  This  is  further  demonstrated  in  Chart  8,  where  we 
have  classed  each  major  category  of  information  discussed  in  our 
interviews  according  to  whether  it  is  almost  always  kept  across 
companies  and  across  projects  over  at  least  some  period  of  time;  whether 
it  is  usually  kept,  that  is,  it  is  kept  for  some  projects,  or  by  some 


ALMOST  ALWAYS 
Process  data  I 
1  Capital  costs  I 


n-maga 


Actual  performance/ 
reliability 

Pioneer  unit 
performance 

Environmental 
attainment _ 

USUALLY 


I  Startup  costs  I 
I  Pioneer  unit  cost  1 
|  Design  issues  and  I 
I  philosophy  j 

I  Equipment  specs/  I 
I  vendors 


IMPORTANCE: 
I  Critical! 
^Important! 
(Secondary]] 


SOMETIMES 

Startup  problems/ 
resolution 

Cost/^  schedule/  I 

performance  problems] 

[Schedule 

Estimates] 


Chart  8  --  Information  Kept  Varies--Even  for  Critical  Data 
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companies,  but  not  all;  and  whether  it  is  sometimes  or  only  rarely  kept. 
It  is  plain  to  see  that  the  information  kept  varies  tremendously,  even 
for  the  critical  data  (critical  means  that  at  least  three-quarters  of 
the  firms  called  it  critical  and  said  they  could  not  make  good  decisions 
without  it). 

Firms  almost  always  capture  and  preserve  critical  information  and 
some  of  the  very  important  information  (described  by  half  or  more  as 
critical,  but  by  some  as  nonessential).  They  usually  keep  some  of  the 
interpretive  information,  especially  the  startup  costs  of  the  pioneer 
unit,  design  issues,  and  so  on. 

But  few  firms  keep  records  on  problems  that  occurred  in  startup, 
how  they  were  resolved,  and  important  information  relevant  to 
understanding  cost,  schedule ,  and  performance  problems.  Understanding 
how  the  pioneer  projects'  actual  cost,  schedule,  and  performance  varied 
from  their  expected  baselines  is  critical  to  good  retrospective 
evaluation  and  investment  decisions.  This  means  that  R&D  and 
engineering  departments'  attempts  to  learn  from  their  past  efforts-- 
and  from  each  other's  efforts--are  constrained  by  a  lack  of 
institutional  memory. 

FACTORS  INFLUENCING  INFORMATION  SURVIVAL 

Our  interviews  of  industry  representatives  yielded  many 
explanations  of  why  the  kinds  of  retrospective  data  we  have  identified 
are  not  always  available.  These  are  some  of  the  comments  we  frequently 
heard . 

"Every  project  is  different."  That  is,  there  being  almost  no  such 
thing  as  a  duplicate  plant,  information  from  a  prior  unit  will  not  be 
useful.  Pioneer  projects  have  unique  problems;  it  is  not  prudent  to 
store  copious  information  on  atypical  problems.  Early  cost  estimates 
are  soon  outdated;  later  ones  are  more  accurate  (which  is  true,  but 
finding  where  the  inaccuracies  were  and  how  they  have  been  updated  is 
important  to  understanding  cost  growth  and  final  cost,  especially  of  a 
pioneer  step).  Many  companies,  especially  the  smaller  ones,  do  not 
build  enough  plants  to  enable  any  kind  of  sophisticated  statistical  work 
or  to  warrant  keeping  cost  and  other  files.  Even  if  people  agreed  that 


they  should  hold  onto  information,  they  may  all  have  different  opinions 
on  what  should  be  kept.  Since  the  firm  cannot  save  everything,  they 
often  end  up  saving  little  or  nothing. 

Data  are  often  not  tracked  very  well  during  a  project  because  it  is 
often  hard  to  know  what  data  will  bo  useful  or  marketable  later. 
Moreover,  technology  can  evolve  so  rapidly  that  extensive  record-keeping 
is  a  waste.  In  these  cases,  firms  tend  to  depend  on  people's  memories. 
Every  company  relies  heavily  on  certain  people  to  keep  and  interpret 
information.  When  they  leave,  the  information  can  lose  its  usefulness. 

Many  interviewees  reported  that  corporate  management  is  not 
sufficiently  committed  to  the  idea  of  keeping  good  information,  even 
though  it  wants  and  uses  it.  Sometimes  information  is  routinely  purged 
after  two  to  seven  years.  When  a  plant  is  shut  down,  many  companies 
have  a  rule  that  all  records  of  the  plant-- including  startup  and  early 
operations--are  destroyed  within  one  or  two  years.  Finally,  to  our 
great  fascination,  we  were  often  told  that  the  data  "wandered  off 
somewhere  and  were  never  seen  again."  The  words  varied,  but  the  story 
was  repeated  many  times. 

How  information  survival  varies  by  company 

Chart  9  summarizes  the  factors  associated  with  information 
survival.  We  found  that  information  capture  and  survival  is  a  function 
of  the  firm's  capability  for  estimating  and  engineering.  Where  there  is 
only  a  small  estimating  shop  or  no  centralized  engineering,  very  little 
information  is  kept  or  retained,  and  the  firm  relies  heavily  on 
contractors  or  other  people.  Where  there  are  extensive  in-house 
estimating  and  engineering  capabilities,  the  firm  retains  and  uses  far 
more  data  in-house.  Such  firms  are  also  better  able  to  interpret  the 
data,  because  they  are  the  ones  who  build  the  plants.  The  resources 
devoted  to  record-keeping  vary  widely,  in  both  money  and  effort. 
End-of-project  reports  are  common,  but  for  the  same  size  project, 
writing  one  may  entail  anywhere  from  half  a  day  to  six  months  of  a 
manager's  time.  They  may  be  tucked  away  in  a  file  drawer  somewhere; 
some  companies  have  rather  sophisticated  or  complex  cost  and  process 
files,  but  most  do  not,  and  hence  there  is  little  systematic  collection 
of  information.  But  even  expensive  data  retention  systems  do  not  always 


capture  and  keep  data  that  the  company  itself  needs  and  could  use. 

Needs  and  uses  also  vary.  If  the  company  does  a  great  deal  of 
engineering,  it  has  an  engineering  standards  "manual"  with  process  and 
mechanical  design  "lessons"  from  major  projects,  especially  pioneer 
plant  developments,  incorporated  into  it  routinely. 

Factors  influencing  information  survival:  Summary 

Information  survival  also  varies  considerably  by  project  even 
within  the  same  company.  For  high-visibility  projects  where  there  is  a 
major  company  stake  or  high-level  sponsor,  the  information  tends  to  be 
kept  more  readily.  The  data  also  tend  to  be  better  kept  if  the  purpose 
of  the  project  is  to  test  a  process  or  demonstrate  a  process  to  be 
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•  Availability  of  project  personnel 


Chart  9  --  Factors  Influencing  Information  Survival:  Summary 


licensed,  as  opposed  to  building  a  more  standard  technology  plant,  and 
if  the  pioneer  plant  is  still  operating.  (When  a  plant  is  shut  down, 
plant  records  are  often  purged  and  the  experienced  people  dispersed,  and 
with  them  the  ability  to  interpret  what  happened  is  lost.) 

The  capture  and  survival  of  information  also  varies  with  the  method 
of  retaining  the  information.  Three  factors  are  primarily  important: 
ease  of  use,  accessibility,  and  completeness.  If  the  information  is 
easy  to  use,  firms  tend  to  collect,  keep,  and  use  more  of  it.  Ease 
implies  standard,  convenient  formats  with  data  recorded  in  common  units 
across  different  projects.  Accessibility  is  important  whether  the 
information  is  contained  in  a  cardboard  box  or  in  a  computer.  Finally, 
information  that  is  complete  is  more  likely  to  survive  because  people 
find  it  more  consistently  useful.  Completeness  implies  more  than  masses 
of  raw  data,  of  course.  An  important  component  is  the  presence  of  the 
information  necessary  to  normalize  the  cost,  schedule,  and  performance 
outcomes  across  projects. 

Finally,  information  survival  depends  to  a  large  extent  upon  people 
themselves.  Consistent  management  commitment  over  time  to  the  capture, 
retention,  and  use  of  retrospective  information  and  analysis  is  vital. 

It  is  also  important  that  the  information  be  useful  to  the  people 
collecting  it  in  the  first  place.  Most  important,  when  information  is 
used  it  tends  to  be  captured  and  preserved  in  ways  that  retain  its  value 
over  time. 

GOOD  COMMUNICATION  AND  PROJECT  MANAGEMENT 

Good  communication  can  be  encouraged  by  good  management. 
Communication  is  enhanced  where  diverse  input  to  project  decisions  is 
encouraged.  This  type  of  information  flow  is  demonstrably  related  to 
better  project  outcomes,  according  to  the  results  of  a  recent  analysis 
of  the  PPS  database.  Especially  in  projects  that  use  innovative 
technology,  involvement  of  a  team  of  representatives  from  the  affected 
corporate  departments  is  desirable.  Such  teams  are  most  effective  when 
their  members  are  recognized  as  having  joint  responsibility  for  the 
project's  success.  Using  a  representative  team  approach  means  bringing 
on  board  all  the  groups  or  divisons  that  will  be  involved  in  the 


project,  and  doing  so  at  the  outset.  These  groups  include  R&D,  process 
development,  engineering,  construction  services,  startup,  and 
operations.  They  should  be  involved  in  the  project  throughout  its  life, 
not  just  when  problems  arise  or  when  their  divison  is  about  to  take  over 
the  project.  Operations  departments  in  particular  should  be  given  a 
stake  in  the  project  and  made  jointly  responsible  for  its  success  from 
the  beginning,  to  reduce  handoff  problems  during  and  after  startup. 

Twenty  of  the  34  projects  for  which  this  information  was  available 
in  the  PPS  database  vested  management  responsibility  for  the  project  in 
such  representative  teams.  This  management  technique  tended  to  be 
associated  with  f irst-of-a-kind  projects;  when  it  wasn't,  the  project 
typically  suffered  higher  cost  growth  and  longer  startup.  Chart  10 
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Chart  10  --  Effect  of  Management  Team  on  Cost  Growth 
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showc  that  cost  growth- -measured  as  the  ratio  of  estimated  costs  as  of 
the  start  of  engineering  to  the  actual  costs  through  startup--averages 
much  worse  for  pioneer  plants  that  did  not  use  this  team  approach 
compared  to  innovative  projects  that  were  directed  by  representative 
teams.  As  Chart  11  demonstrates,  failure  to  use  a  team  approach  for 
pioneer  plant  projects  is  associated  with  startup  times  an  average  of 
three  times  longer  than  those  of  pioneer  plants  that  used  a  team 
approach.  In  both  cases,  the  team  approach  does  not  seem  to  make  as 
much  difference  for  plants  using  relatively  proven  technologies. 
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Chart  11  --  Effect  of  Management  Team  on  Startup  Time 
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Charts  12  and  13  present  the  results  of  our  analysis  of  project 
management,  after  controlling  for  other  factors,  such  as  innovation, 
project  definition,  and  feedstock  type,  that  explain  project  outcomes.4 
Cost  growth  is  explained  by  the  level  of  project  definition  and  the 
proportion  of  the  estimated  cost  in  steps  involving  new  technology.  The 
model  further  illustrates  the  effect  of  not  using  a  representative  team 
approach  for  pioneer  plants.  It  predicts  that  the  engineering  estimate 
will  probably  fall  more  than  11  percentage  points  below  actual  costs, 
after  controlling  for  the  effects  of  project  definition  and  innovation. 


1.141  CONSTANT 

-0.072  x  Level  of  project  definition 

-0.003  x  Percent  of  capital  in  new  technology 

-0.115  IF  representative  team  approach  not  used 
(pioneer  plants  only) 

R*  =  .67 

Standard  error  =  0.1 1 
Number  of  plants  =  32 


Chart  12  --  A  Model  of  Cost  Growth  with  Management  Team 

4The  variables  shown  in  Charts  12  and  13  are  defined  in: 
R-2569-D0E,  (Morrow,  Phillips,  and  Myers),  op.  cit. 


Startup  time,  from  introduction  of  feed  through  relatively  steady 
operations,  is  explained  by  three  factors:  innovation,  measured  as  the 
number  of  functional  process  blocks  involving  new  technology,  the  type 
of  feedstock,  and  management  approach.  Starting  with  an  intercept  of 
about  0.4,  expected  startup  time  in  months  is  predicted  as  follows:  ad 
almost  3  months  for  each  process  step  that  involves  commercially 
unproven  technology;  add  about  half  a  year  if  the  feedstock  is  a  solid 
material;  then  add  another  7  or  8  months  if  a  representative  team 
approach  is  not  used.  The  effect  of  this  last  management  factor 
suggests  the  importance  of  including  R&D  personnel  throughout  the 
project  life,  and  of  including  operations  people  early  in  the  project-- 
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+2.84  x  New  steps 
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+7.70  IF  representative  team  approach  not  used 
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Chart  13  --  A  Model  of  Startup  Time  with  Management  Team 


and  giving  both  groups  a  share  of  the  responsibility  for  project 
success . 

LESSONS  FOR  COMMUNICATION  AND  TECHNOLOGY  COMMERCIALIZATION 

By  drawing  on  the  results  of  recent  Rand  research,  we  have  seen 
empirical  support  for  what  might  have  seemed  obvious--except  that  it  is 
so  often  ignored:  Sucessful  commercialization  of  new  technologies 
requires  good  communication  and  information  flow  among  R&D,  engineering, 
operations,  and  management  teams. 

Structuring  project  management  decisionmaking  to  facilitate  diverse 
inputs  from  the  affected  corporate  divisions  pays  significant, 
quantifiable  dividends.  When  new  technologies  are  being  commercially 
introduced,  representatives  of  R&D  and  operations  should  be  included 
from  the  project  outset.  This  practice  is  associated  with  lower  cost 
growth  and  shorter  startup  times. 

Information  policies  and  practices  commonly  employed  by  the  process 
industries  fail  to  meet  specific,  recognizable  needs.  Information 
reflecting  "lessons  learned"  from  pioneer  projects  is  not  thoroughly 
collected  (or  "captured")  in  the  first  place  by  most  firms.  When  it  is, 
what  is  captured  is  often  not  easily  available  to  those  who  need  it,  or 
shared  by  those  who  have  it.  To  make  matters  worse,  information  tends 
not  to  last  very  long.  Valuable  information  is  often  purged  routinely 
after  even  a  very  short  time;  what  is  kept  is  not  well  maintained. 

Two  particularly  valuable  types  of  information  are  not  regularly 
captured  and  made  available.  These  represent  the  essence  of  what  could 
be  learned  during  the  commercialization  process:  the  pilot  plant 
experience  preceeding  the  pioneer  unit,  and  startup  and  early  operations 
of  the  pioneer  commercial  plant.  Both  are  vital  to  understanding  the 
new  process,  and  remembering  what  happened  is  critical  to  good 
decisionmaking  by  R&D,  engineering,  operations,  and  management.  In  sum, 
not  remembering  what  happened  makes  it  extremely  difficult  to 
communicate . 
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